Use it or Lose it: Selective Memory and Forgetting in a Perpetual Learning Machine
نویسنده
چکیده
In a recent article we described a new type of deep neural network– a Perpetual Learning Machine (PLM) – which is capable of learning ‘on the fly’ like a brain by existing in a state of Perpetual Stochastic Gradient Descent (PSGD). Here, by simulating the process of practice, we demonstrate both selective memory and selective forgetting when we introduce statistical recall biases during PSGD. Frequently recalled memories are remembered, whilst memories recalled rarely are forgotten. This results in a ‘use it or lose it’ stimulus driven memory process that is similar to human memory.
منابع مشابه
Effects of Citalopram on Learning and Memory in the Mouse and Rat
Data on the effects of serotonin reuptake inhibitors on learning and memory processes are not consistent. In the present study, the effects of citalopram, a very potent and completely selective inhibitor of the serotonin reuptake on spatial discrimination in the T-maze and Morris water maze, were assessed in mice and/or rats. Animals received different doses of citalopram (1, 2, 4, 8 or 16 mg/k...
متن کاملOn-the-Fly Learning in a Perpetual Learning Machine
Despite the promise of brain-inspired machine learning, deep neural networks (DNN) have frustratingly failed to bridge the deceptively large gap between learning and memory. Here, we introduce a Perpetual Learning Machine; a new type of DNN that is capable of brain-like dynamic ‘on the fly’ learning because it exists in a self-supervised state of Perpetual Stochastic Gradient Descent. Thus, we ...
متن کاملAbstraction is Harmful in Language Learning
The usual approach to learning language processing tasks such as tagging, parsing, grapheme-to-phoneme conversion, pp-attachrnent, etc., is to extract regularities from training data in the form of decision trees, rules, probabilities or other abstractions. These representations of regularities are then used to solve new cases of the task. The individual training examples on which the abstracti...
متن کاملSelective memory retrieval can impair and improve retrieval of other memories.
Research from the past decades has shown that retrieval of a specific memory (e.g., retrieving part of a previous vacation) typically attenuates retrieval of other memories (e.g., memories for other details of the event), causing retrieval-induced forgetting. More recently, however, it has been shown that retrieval can both attenuate and aid recall of other memories (K.-H. T. Bäuml & A. Samenie...
متن کاملDual-Network Memory Model for Temporal Sequences
In neural networks, when new patters are learned by a network, they radically interfere with previously stored patterns. This drawback is called catastrophic forgetting. We have already proposed a biologically inspired dual-network memory model which can much reduce this forgetting for static patterns. In this model, information is first stored in the hippocampal network, and thereafter, it is ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1509.03185 شماره
صفحات -
تاریخ انتشار 2015